Soroban: Attributing Latency in Virtualized Environments

Applications executing on a hypervisor or in a container experience a lack of performance isolation from other services executing on shared resources. Latencysensitive applications executing in the cloud therefore have highly-variable response times, yet attributing the additional latency caused by virtualization overheads on individual requests is an unsolved problem. We present Soroban, a framework for attributing latency to either the cloud provider or their customer. Soroban allows developers to instrument applications, such as web servers to determine, for each request, how much of the latency is due to the cloud provider, and how much is due to the consumer’s application or service. With this support Soroban enables cloud-providers to provision based on acceptable-latencies, adopt finegrained charging levels that reflect latency demands of users and attribute performance anomalies to either the cloud provider or their consumer. We apply Soroban to a HTTP server and show that it identifies when the cause of latency is due to a provider-induced activity, such as underprovisioning a host, or due to the software run by the customer.